Detecting Anomalous Cybersecurity Events Using LSTM Autoencoders

Introduction

• Modern infrastructures face sophisticated, process-level intrusions that mimic benign behavior.
• Static rule-based detection fails to capture temporal dependencies.
• We investigate LSTM-based anomaly detection for process logs.
• Key idea: model normal system activity and detect deviations as anomalies.

LSTM Motivation

• LSTM networks (Hochreiter et al. 1997):
  • Solve vanishing gradient problem with gated memory cells.
  • Capable of learning long-range temporal dependencies.
• Applications:
  • Speech recognition (Graves et al. 2013).
  • Language modeling (Sundermeyer et al. 2012).
  • Clinical anomaly detection (Lipton et al. 2016).
• Why relevant ?
  • Cybersecurity logs are sequential — order and timing matter.

Cybersecurity Context

• Kim et al. 2016: LSTM classifiers for intrusion detection outperform static baselines.

• Malhotra et al. 2016: LSTM encoder-decoder reconstructs normal sequences, anomalies detected by high reconstruction error.

• Yin et al. 2017: RNNs generalize better on raw network flows.

• Cinque et al. 2022: Micro2vec + LSTM captures log anomalies in microservices.

Data Challenge

• Old datasets (KDD’99, NSL-KDD, ISCX 2012) are synthetic and outdated.

• BETH dataset (Highnam et al. 2021)(inproceedings?):

  • 8M+ process-level events (eBPF-instrumented honeypots).
  • Rich features: timestamps, syscalls, labels (benign vs. malicious).
  • Realistic temporal and adversarial patterns for modern systems.

Our Approach

• Semi-supervised LSTM Autoencoder:
  • Train only on benign sequences (normality modeling).
  • Use reconstruction error to flag anomalies.
• Goal: detect process-level attacks in test logs.

Methods

train_dataset.csv already exists - loading from disk
validation_dataset.csv already exists - loading from disk
test_dataset.csv already exists - loading from disk

Understanding LSTM Networks

• Recurrent Neural Network (RNN) with memory
• Keeps track of long sequences
• Ideal for logs, time series, and system calls

LSTM Fundamentals

• Forget Gate:
  \(f_t = \sigma(W_f [h_{t-1}, x_t] + b_f)\)

• Input Gate and Candidate Update:
  \(i_t = \sigma(W_i [h_{t-1}, x_t] + b_i)\)
  \(\tilde{C}_t = \tanh(W_c [h_{t-1}, x_t] + b_c)\)

• Cell State Update:
  \(C_t = f_t * C_{t-1} + i_t * \tilde{C}_t\)

• Output Gate and Hidden State:
  \(o_t = \sigma(W_o [h_{t-1}, x_t] + b_o), \;\; h_t = o_t * \tanh(C_t)\)

Figure 1: A module of LSTM network (Trinh et al. 2021)

LSTM Autoencoder for Anomaly Detection

Figure 2: LSTM Autoencoder for Anomaly Detection (Trinh et al. 2021)

Anomaly Detection from Reconstruction Error

The BETH Dataset

• Dataset size: 8,004,918 system events.
• Training subset: ~763,145 benign events (60/20/20 split).

Data Preprocessing and Sequence Modeling

Preparing the Data

• Feature Selection

• Encoding:
  One-hot encoding for categorical features

• Sequence Generation: sliding window and Reshaped logs into 3D tensors

• Sequence Labeling:
  Normal: Only benign events
  Anomalous: At least one suspicious (Sus) or malicious (Evil) event

LSTM Training & Architecture

Trained on normal sequences only

Symmetric autoencoder: stacked LSTM + bottleneck

Optimized via Adam, minimizing reconstruction loss

Generalization: early stopping, dropout, 5-fold CV

Hyperparameters tuned based on prior work (Nguyen et al. 2021; Malhotra et al. 2016).

Threshold Selection and Evaluation Metrics

Threshold selection via:

• 95th percentile of training errors
  
• F1-score optimization (validation set)
  
• Joint minimization of FP/FN rates

Evaluation metrics:

• Precision, Recall, F1-score
  
• ROC-AUC, Confusion Matrix
  
• Error distribution plots

Data Exploration and Visualization

Event Frequency Distribution.

Process Behavior

Temporal Skew in Data Collection

Label Imbalance and Anomaly Prevalence

Correlation

Correlation

Boxplot of entropy values

Training and Validation Performance

• 5-fold cross-validation:

  • Avg. validation loss: 0.1516 ± 0.1443.

• Stable convergence.

Figure 7: Training / validation Loss result

Anomaly Detection Threshold

Distribution of reconstruction error separates normal vs. anomalous.

Figure 8: Histogram of test reconstruction errors (MAE)

Classification Metrics

Precision: 99.3%, Recall: 99.5%, F1-score: 99.4%.

Figure 9: Precision Recall, F1 vs threshold plot

Confusion Matrix

Figure 10: Confusion matrix for final predictions

ROC Curve

ROC AUC = 99.5%.

Figure 11: ROC Cuve